Method of color image processing to eliminate shadows and reflections

ABSTRACT

A method of color image processing comprises at least the following steps: the color image is converted into an intermediate image comprising components that depend solely on the H and S components of the original image in an HSV or HLS representation; the intermediate image comprising at least two components X and Y, determined by functions especially of the H component, these functions taking the same value when the H component is zero or equal to one; a new image comprising only one component is generated from the intermediate image, the component of this new image being a function of the components of the intermediate image. The disclosed method can be applied especially to the heightening of contrast in remote surveillance, and to image processing before segmentation in mobile robotics.

RELATED APPLICATIONS

The present application is based on International Application No.PCT/EP2003/050403, filed Sep. 12, 2003, and claims priority from, FrenchApplication Number 0211684, filed Sep. 20, 2002, the disclosure of whichis hereby incorporated by reference herein in its entirety.

BACKGROUND OF THE INVENTION

The present invention relates to a method for the processing of colorimages to eliminate shadows. It can be applied especially to theheightening of contrast in remote surveillance, and to image processingbefore segmentation in mobile robotics.

In applications of mobile robotics, operations of image segmentation arecarried out. Segmentation makes it possible to detect and localize theedges of roads for example. The shadows of trees on these roads orreflections from puddles generate parasitic segments. These shadows andreflections constitute parasitic information. The invention makes itpossible to eliminate this parasitic information by carrying out aprocessing of color images that are to be segmented.

SUMMARY OF THE INVENTION

An object of the invention is a method of color image processing,comprising the following steps:

-   -   the color image is converted into an intermediate image        comprising components that depend solely on the H and S        components of the original image in an HSV or HLS        representation,    -   the intermediate image comprising at least two components X and        Y, determined by functions especially of the H component, these        functions taking the same value when the H component is zero or        equal to one;    -   a new image comprising only one component is generated from the        intermediate image, the component of this new image being a        function of the components of the intermediate image.

The method according to the invention has many advantages:

-   -   it can be used to extract the useful information from a color        image with a minimum loss of information;    -   it can be used to compress the information in a monochromatic        image;    -   it is totally non-parametrical and can therefore be applied to        all images with the same efficiency, without any particular        setting, whatever the scene.

After a processing operation of this kind, it is possible to usemonochromatic image segmentation algorithms while at the same timeexploiting the wealth of information of a color image in getting rid oferrors due to shadows or reflections of water films.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention shall now be described in great deal in the context of aparticular example of a practical implementation. In this description,reference shall be made to the figures of the appended drawings, ofwhich:

FIG. 1 represents the RGB color space;

FIG. 2 represents the HSV color space;

FIG. 3 is a graph showing the steps of implementation of the inventionaccording to an advantageous mode using texture attributes;

FIG. 4 exemplifies a result of a Karhunen-Loève transformationoperation;

FIG. 5 shows an alternative implementation of the example shown in FIG.3;

FIG. 6 is an exemplary histogram of a new image obtained by theimplementation of the invention to which a filtering of pixels and anadjustment of the dynamic range is applied;

FIG. 7 shows another exemplary adjustment of the dynamic range.

MORE DETAILED DESCRIPTION

Referring to FIG. 1, a color image is generally described in the RGB(red, green, blue) space. The color of each pixel is represented thereinby three components R (red), G (green), B (blue). These components maybe represented on three axes 11, 12, 13 in Cartesian coordinates. Theprimary colors, cyan, blue, magenta, red, yellow and green, form thecorners of a cube 10 in this space. Black and white form oppositecorners of this cube. The segment connecting these points is a maindiagonal of this cube on which only the luminance L varies.

Referring now to FIG. 2, other color spaces exist. In the HSV (hue,saturation, value) space, color is represented by three components H, S,and V. These components may be represented in a referential system ofcylindrical coordinates. The H component is represented by an angle 21.The S and V components are represented on two axes 22 and 23. The Hcomponent is the hue. This component makes it possible for example todistinguish red and yellow. The S component is saturation, namely thepurity of the color. This component makes it possible for example todistinguish red and pink. The V component is the value or intensity,namely the quantity of light. This component makes it possible, forexample, to distinguish between dark gray and light gray, or againbetween dark red and light red. It is assumed hereinafter in thedescription that the components H, S, V are normalized between 0 and 1.The following explanations may be transposed in an obvious way if thesecomponents are not normalized between 0 and 1.

The passage from the RGB representation to the HSV representation isdone according to known analytical formulae. The primary colors andblack form a six-faced pseudo-cone (hexcone) 20. The plane with aconstant value of V corresponds to the projection of the RGB cubeparallel to its main diagonal.

The Applicant has noted that the value or intensity V does not carry agreat deal of useful information because the elements of the image to besegmented are distinguished far more by their spectral properties thanby their reflection or their total absorption. Furthermore, the value orintensity V carries information that leads the segmentation algorithmsinto error, since shadowed and non-shadowed zones of a same object (aroad for example) have very different intensities (with variation ofintensity at the source). Similarly, a water film on the road gives riseto a strong reflection and, therefore, a very different intensity fromthat of a dry or simply wet portion on the same road (with variation ofintensity by modification of the properties of reflection).

In order to eliminate the information on value or intensity, the colorimage is converted into an intermediate image in which the V componentis absent. This intermediate image includes components that dependsolely on the H and S components of the original image in an HSVrepresentation.

This makes it possible to eliminate cast shadows and reflections due towater films. This furthermore has the advantage of reducing the quantityof information present in the image (by the elimination of a component).

The conversion from the RGB space to the HSV space gives rise tosingularities of hue for certain points. Thus, for the achromatic points(R=G=B), a small variation in RGB may lead to a big variation in H.Furthermore, for the points in the vicinity of the axis of origin of H(G=B, R>(G+B)/2), a small disturbance in the components R, G, B may leadto major variations in H (a change in the sign of H).

Furthermore, the distribution of the converted values is not uniform.Digitization may lead to peaks and holes in the distribution of thevalues. In other words, the information is noise-ridden around V=0.

Advantageously, components X and Y are used in the intermediate image.These components are determined by functions especially of the Hcomponent, and take the same value when the H component is zero or equalto one. Thus, the similarities induced by the H component, proper to thesystem of cylindrical coordinates, are eliminated.

These functions are written as G_(X)(H) and G_(Y)(H). They thereforeverify the following relationships: G_(X)(0)=G_(X)(1) andG_(Y)(0)=G_(Y)(1). The components X and Y are then determined by:X=G _(X)(H)Y=G _(Y)(H)

For example, the following functions may be used:G _(X)(H)=cos(2πH−φ)G _(Y)(H)=sin(2πH−φ)where φ is a constant.

A third component can be added to this intermediate image: saturation S.Thus, in the intermediate image, there is hue information H andsaturation information S, while at the same time the singularities havebeen eliminated.

Advantageously, it is possible to combine hue information H andsaturation information S in only two components of the intermediateimage. To this end, the components X and Y are determined by functionsnot only of the H component but also of the S component. In the same wayas above, these functions take the same value when the H component iszero or equal to one. Furthermore, these functions tend toward zero whenthe S component tends toward zero.

These functions are written as G′_(X)(H,S) and G′_(Y)(H,S). Theytherefore verify the following relationships:

$\begin{matrix}\begin{matrix}\begin{matrix}{{G_{X}^{\prime}\left( {0,S} \right)} = {G_{X}^{\prime}\left( {1,S} \right)}} \\{{G_{Y}^{\prime}\left( {0,S} \right)} = {G_{Y}^{\prime}\left( {1,S} \right)}}\end{matrix} \\\left| \begin{matrix}{G_{X}^{\prime}\left( {H,S} \right)} & \rightarrow & 0 \\S & \rightarrow & 0\end{matrix} \right.\end{matrix} \\\left| \begin{matrix}{G_{Y}^{\prime}\left( {H,S} \right)} & \rightarrow & 0 \\S & \rightarrow & 0\end{matrix} \right.\end{matrix}$

For example, it is possible to use a product of two independentfunctions respectively of the S component and of the H component:G′ _(X)(H,S)=F _(X)(S)×G _(X)(H)G′ _(Y)(H,S)=F _(Y)(S)×G _(Y)(H)

The functions F_(X)(S), F_(Y)(S) of the S component are functionstending toward zero when the S component tends toward zero. Thesefunctions may be, for example, the identity function (for exampleF_(X)(S)=S) or the square function (for example F_(X)(S)=S²).Preferably, monotonic and continuous functions will be used.

Referring now to FIG. 3, in an advantageous variant, an intermediateimage 34 is generated. This intermediate image furthermore comprisestexture attributes. The texture attributes are determined from an image32 comprising the components X and Y for example, and not from theinitial color image 30 encoded in the RGB space. Indeed, it is soughtfirst of all to eliminate the shadows and the reflections.

For example the initial color image 30 is converted into an image 31with two components, the H and S components. Then, the image 31 isconverted into an image 32 that comprises the two components X and Ydefined here above. For each pixel of the image 32, texture attributesare determined. These attributes may be, for example, the attributesdefined by Haralick. Of course, other attributes may be used, through afractal approach for example. An image 33 is obtained with textureattributes. The components X and Y and the texture attributes arecombined to form the intermediate image 34, to which the remainingprocessing operations are applied. It is thus possible to apply amultitude of transformations to the intermediate image without beingpreoccupied by shadows and reflections.

In order to obtain a monochromatic image, a new image is generated fromthe intermediate image obtained. In other words, a transformation isapplied to the intermediate image so as to keep only one component. Thesingle component of the new image is a function of the components of theintermediate image.

Referring now to FIG. 4, in an advantageous embodiment of the invention,the new image is generated by keeping only the first component of theKarhunen-Loève transformation.

This transformation is applied in the referential system D1, D2 of theeigen vectors are associated with the covariance matrix of the pixels ofthe intermediate image. There is thus a passage from the X, Yreferential system to the D1, D2 referential system. As can be seen inthe exemplary distribution of the pixels 40 of the image in thesereferential systems, this transformation enables the decorrelation ofthe information present in the image. The first component D1 is the mostdiscriminatory component, i.e. it is this component that maximizes theinformation. This facilitates discrimination in the segmentationprocessing operations applied to the new image.

According to an alternative mode of implementation, a linearapproximation of the Karhunen-Loève transformation is applied, not theKarhunen-Loève transformation itself. To this end, the Karhunen-Loèvetransformation is applied to a representative set of images (for exampleeight images). The images of this set have characteristics similar tothe images to which it is sought to apply the linear approximation. Anaverage is then taken of the changes in referential system obtained bythe exact Karhunen-Loève transformation. A average change of referentialsystem is obtained, and this referential will be used as anapproximation of the Karhunen-Loève transformation.

According to another mode of implementation, the new image is generatedby projecting the components of the intermediate image in the plane inwhich the dynamic range or the mean standard deviation is the greatest.

Reference is now made to FIG. 5 which represents an alternative mode ofimplementation with respect to the example shown in FIG. 3. In the sameway, from the initial color image 50, an image with two components, theH and S components, is generated. Then an image 52 comprising the twocomponents X and Y is generated. Then the Karhunen-Loève transformation(or a linear approximation of this transformation) is applied to thecomponents X and Y. The first component of the result of thistransformation is kept to form an image 53 comprising a singlecomponent. Texture attributes of this image 53 are determined. Thesetexture attributes form an image 54. The single component of the image53 and the texture attributes (image 54) are combined to form theintermediate image 55 comprising color information and textureinformation. The remaining processing operations are applied to thisintermediate image 55.

The Karhunen-Loève transformation can thus be applied a second time, forexample to obtain a monochromatic image.

Reference is now made to FIG. 6 which represents an exemplary histogram60 of the component of the monochromatic image. The x-axis 61 representsthe values taken by the pixels. The y-axis 62 represents the number ofpixels in the image having a determined value. A non-negligible range ofthe values is taken up by pixels that are very small in number at thetwo extreme ends 63 a, 63 b of the histogram.

Advantageously the lightest and the darkest pixels, which are present invery small numbers, are filtered. This makes it possible to avoid anyloss of dynamic range through the occupying of a major part of the rangeof values by pixels that carry no additional information. In otherwords, the darkest and the lightest pixels, which represent a determinedfraction of the total number of pixels of the image, are filtered. Tothis end, a number of pixels to be eliminated from the image isdetermined. This number of pixels may be expressed as a percentage ofthe total number of pixels of the image. A minimum value VMIN and amaximum value VMAX of the pixels are determined, so that the number ofpixels of the image whose value is below VMIN or above VMAX is as closeas possible to the number of pixels to be eliminated. For all the pixelswhose value is beyond the values VMIN and VMAX, the value of thecorresponding limit is assigned.

The values VMIN and VMAX may be determined iteratively. During aniteration, a value 64 of the y-axis 62 is selected in going through theaxis from the bottom to the top. The smallest value VMIN and thegreatest value VMAX of the y-axis 61, corresponding to the intersections64 a, 64 b of the histogram 60 with the selected value 64, aredetermined. The operation is stopped when the number of pixels to beeliminated for the selected value is greater than the determinedfraction of the total number of pixels of the image.

The values taken by the component of the monochromatic image (the mostdiscriminatory component) are distributed in a non-normalized range. Inpreparation for subsequent processing operations (such as segmentation,viewing, etc.), these values may be brought by linear transformationinto a normalized interval, for example [0.1], or [0.255]. This lineartransformation 65 may be applied, as illustrated in FIG. 6, after thepixels have been filtered in small numbers at the two limits of thehistogram. Thus, for an image encoded on 8 bits, the value VMIN isbrought to a level zero, and the value VMAX is brought to a level 255.

Referring now to FIG. 7, an advantageous variant is described of theadjustment of the values used to process the image by giving greater orlesser importance to the contrast in a selected zone.

In this advantageous variant, the beginning of the processing is carriedout in a similar way. The parameters of the Karhunen-Loèvetransformation (or a linear approximation of this transformation) aredetermined according to the selected zone of the image only. Thistransformation is then applied to the entire image. The pixels of theselected zone occupy only a part 70 of the histogram 60.

Similarly, a linear transformation can be applied to bring the values ofthe pixels into a normalized interval. Advantageously, the parameters ofthe linear transformation are determined so as to bring the minimumvalue VMIN and the maximum value VMAX of the selected zone between twopredetermined levels NMIN and NMAX. These levels NMIN and NMAX arestrictly contained in the normalized interval. For example, for an imageencoded on 8 bits, the normalized interval being [0.255], a value of 32can be taken for NMIN and 224 for NMAX.

These parameters are used to apply a linear transformation to the entireimage, the levels of the pixels being limited (for pixels outside theselected zone) in order to be included in the normalized interval.

This advantageous variant is used to obtain efficient discrimination inthe regions similar to the selected zone, while at the same time keepingother information out of this selected zone.

1. A method for converting an original color image into a monochromatic image said conversion being made with a minimal lose of information, the method comprising the following steps: a computer for performing the steps of: converting the original color image made of pixels, each pixel being represented by hue H, saturation S and value V components in a color space, or by hue H, luminescence L and saturation S components in a HLS color space into an intermediate image made of pixels, each pixel being represented by components X and Y depending solely on the H and S components of the pixel in the original color image; wherein components X and Y are functions of the H component determined by: X=G _(X)(H) and Y=G _(Y)(H); wherein these functions verify the following relationships: G _(X)(0)=G _(X)(1) and G _(Y)(0)=G _(Y)(1); (b) generating a new monochromatic image from the intermediate image; wherein the intermediate image each pixel is represented only by a component D1; D1 being a function of components X and Y of the pixel in the intermediate image.
 2. The method according to claim 1, wherein components X and Y are functions of the H and S components determined by: G′ _(X)(H,S) and G′ _(Y)(H,S), these functions verify the following relationships: G′ _(X)(0,S)=G′ _(X)(1,S) and G′ _(Y)(0,S)=G′ _(Y()1,S) G′ _(X)(H,S)→0 and G′ _(Y)(H,S)→0 S→0 S→0.
 3. The method according to claim 2, wherein G′_(X)(H,S) and G′_(Y)(H,S) are monotonic and continuous functions of the S component.
 4. The method according to claim 1, wherein the new monochromatic image is generated by applying a Karhunen-Loève transformation, or a linear approximation of this transformation, to the intermediate image and using only the most discriminatory component D1 to represent each pixel of the new image.
 5. A method for processing an original color image, the method comprising the following steps: a computer for performing the steps of: (a) converting the original color image into an intermediate image made of pixels, each pixel having components that depend solely on hue H and saturation S components of the pixels in the original color image in an HSV or HLS representation; wherein the intermediate image is made of pixels, each pixel comprising two components X and Y, determined by functions of the H component, theses functions take the same value when the H component is zero or equal to one; (b) generating a new monochromatic image from the intermediate image, said new monochromatic image being made of pixels, each pixel having only one component which is a function of the components of the corresponding pixel in the intermediate image; wherein the new monochromatic image is generated by projecting the components of the pixels of the intermediate image in a plane in which a dynamic range or mean standard deviation is the greatest.
 6. The method according to claim 1, wherein a filtering is performed on the darkest and the lightest pixels, which represent a determined fraction of the total number of pixels of the new monochromatic image.
 7. The method according to claim 5, wherein the dynamic range of the new image is adjusted to a total available dynamic range.
 8. The method according to claim 1, wherein G_(X)(H) and G_(Y)(H) are defined by: GX(H)=cos(2πH−φ) and GY(H)=sin(2πH−φ); where φ is a constant.
 9. The method according to claim 3, wherein the new monochromatic image is generated by applying the Karhunen-Loève transformation, or a linear approximation of this transformation, to the intermediate image and using only the most discriminatory component D1 to represent each pixel of the new image.
 10. The method according to claim 3, wherein the new image is generated by projecting the components of the pixels of the intermediate image in the plane in which the dynamic range or mean standard deviation is the greatest.
 11. The method according to claim 1, wherein the new monochromatic image is generated from a combination of an image made of pixels having components X and Y with additional texture attributes of the image having components X and Y. 